Non-Linear Dimensionality Reduction: A Comparative Performance Analysis

We present an analysis of the comparative performance of non-linear dimensionality reduction methods such as Non-Linear Mapping, Non-Metric Multidimensional Scaling and the Kohonen Self-Organising Feature Map for which data sets of diierent dimensions are used. To obtain comparative measures of how well the mapping is performed, Procrustes analysis, the Spearman rank correlation coeecient and the scatter-plot diagram are used. Results indicate that, in low dimensions, Non-Linear Mapping has the best performance especially when measured in terms of the Spearman rank correlation coeecient. The output from the Kohonen Self-Organising Feature Map is easier to interpret than the output from the other methods as it often provides a superior qualitative visual output. Also, the Kohonen Self-Organising Feature Map may outperform the other methods in a high-dimensional setting. In many applications, dimensionality reduction is used to explore a data set to try to obtain some insight into the nature of the phenomenon that produced the data. We are often interested in understanding the structural relationships that exist in the feature space, such as clusters or data point density discontinuities. Measures that could be used to reveal such structural relationships include the inter-point distance, the \shape" of the data distribution etc: : :Such an understanding is very domain-dependent and may require a deep understanding of the structures, causality etc: : :that may exist between the features. In some cases, the application may impose a constraint on the serialisation or ordering of the topological mapping as in, for example, chronological ordering in a time series or regression analysis CLN91]. In many applications, one important objective of dimensionality reduction is that it preserves as much as possible the structural relationships that exist in the data set when 1996 Springer-Verlag.